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Abstract. Motions of virtual characters in movies or video games are 
typically generated by recording actors using motion capturing methods. 
Animations generated this way often need postprocessing, such as improv¬ 
ing the periodicity of cyclic animations or generating entirely new motions 
by interpolation of existing ones. Furthermore, search and classihcation 
of recorded motions becomes more and more important as the amount of 
recorded motion data grows. 

In this paper, we will apply methods from shape analysis to the process¬ 
ing of animations. More precisely, we will use the by now classical elas¬ 
tic metric model used in shape matching, and extend it by incorporating 
additional inexact feature point information, which leads to an improved 
temporal alignment of different animations. 

Keywords. Curve matching, feature point information, skeletal anima¬ 
tion, shape analysis, square root velocity transform. 
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1 Introduction 

Animations. Virtual characters in movie and TV special effects or video games are 
most commonly animated using skeletal animation, where a character’s motion is de¬ 
scribed in terms of joint-angles in an approximation of a human skeleton. Motion 
capturing is a typical way to generate such animations, whereby an actor or stuntman 
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performs the requested motion while being recorded from multiple angles. From this, 
the underlying skeletal pose can then be reconstructed. While motion capturing can 
produce very life-like animations, it has a number of drawbacks, some of which can be 
addressed algorithmically uniiz]- 

In [10] , methods from shape analysis were applied to a number of problems in com¬ 
puter animation: periodicity of animations, interpolation between animations and mo¬ 
tion recognition. A central component in this is the modelling of curve reparametriza- 
tions as elements of the diffeomorphism group on the circle or the unit interval, Diff(S'^) 
and Diff([0,1]) respectively. For animations, such reparametrizations can be used to 
align them in time, similar to the concept of timewarp curves in [121113] . Note that 
the skeletal animation approach is very different in concept and applications from 
silhouette-based representations of motions, as for example in [T]. 

In this paper we will demonstrate how feature point information can be incorporated 
into shape matching techniques, and how this can be applied in the context of computer 
animations in order to improve the temporal alignment of related actions. 

Shape matching. The field of shape analysis concerns itself with the study and clas¬ 
sification of similarities and dissimilarities within certain classes of shapes. In order to 
achieve this objective, a variety of different methods has been developed, each tailored 
to the actual class of shapes under consideration. A particularly important example 
for a shape space, which is also the main focus of this article, is the space of un¬ 
parametrized curves^ and, in recent years, Riemannian methods for shape spaces of 
curves have been deeply explored both theoretically [291 ng EH na nn Ea ng ESI 016] 
as well as from an application oriented point of view [HEIllIillSg- See a for an 
overview of these topics. 

Although the main contribution of this article is not limited to a specific Riemannian 
metric on the shape space of unparametrized curves, we will focus for simplicity on a 
particular one that is related to the so called Square Root Velocity (SRV) framework 
|25l|3]. This metric is given as the pullback of the L^-metric via a transformation called 
the SRVT. This allows for the development of extremely efficient numerical methods 
and, as a consequence, it has been successfully used in a series of applications EHiiin]. 
See also (snug for other reparametrization invariant metrics that can be represented 
in a similar way. 

Our main addition to this standard setting will be the incorporation of point cor¬ 
respondences between two curves one wants to match. In the particular application 
of animation processing, these correspondences will describe similar poses at different 
points in time in the two animations. We assume here that the point correspondences 
are manually entered pieces of information that are possibly error-prone. Thus we are 
interested in an exact matching of the unparametrized curves, but only an inexact 
matching of the point correspondences. Our approach differs from the one presented 
in EZ], where feature information is included in the form of auxiliary functions that are 
combined with the geometric curve to form higher dimensional curves that can then be 
matched by the usual methods. Instead, we augment the matching energy functional 

^Closed curves can be used, for example, to represent outlines of objects in recognition applications. 
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with an extra feature matching term, leaving the elastic matching terms unchanged. 
In comparison to |14j . which focused on matching three dimensional surfaces, we al¬ 
low inexact matching of landmarks in our model. We will discuss in this article both 
the theoretical framework of shape matching with feature points and computational 
aspects. 

Overview In Sect.j^we give an overview of the general framework for shape matching 
with additional feature point information and will then apply this framework to one 
particular choice of metric: the metric that is induced by the SRV transformation. 
We then present the basics of two algorithms that can be used to determine the 
optimal parametrization to match two curves: In Sect. [XT] we derive all the formulas 
necessary to find the parametrization using a gradient descent approach. In addition, 
in Sect. |3.2[ we discuss the application of a dynamic programming based algorithm. In 
Sect. 1^ we apply the previously presented framework to process human motion data. 
Moreover, we show some examples of an application to two-dimensional curves, where 
the workings of the method can be better visualized. 

2 The general framework 

Problem formulation. We will start this article by formulating our main problem: 

Problem 1. Given two unparametrized curves [cq] and [ci] and a number of n point 
correspondences between the curves - i.e., points Cq and C\ that should be matched 
onto each other - we want to find an optimal deformation between the two curves, that 
also respects the alignment of the feature points. 

In order to achieve this goal, we will first choose representatives of the shapes [cq] and 
[ci], i.e., parametrized curves cq and ci. Each representative Cj determines parameter 
values 6j that correspond to the feature points (7* in the sense that 



Thus we can represent any shape [cq] with additional feature point information as a tu¬ 
ple (co, 00 )5 with Co S Imm(5'^,IR'^) and 6q = (0g)i £ (5^)". One way to tackle Problem 
[^is then to construct a similarity measure on the product space Imm(S'^,K‘^) x 
that has certain invariance properties with respect to the action of the diffeomorphism 
group. 

To do this mathematically rigorously we will need some results and definitions from 
infinite dimensional Riemannian geometry first. 

2.1 The manifold of parametrized curves 

In this article we will consider the space of regular curves from a parameter space M 
into some - possibly high dimensional - K'^: 

Imm(M,M‘^) := {c £ : |c'| > 0} . (1) 
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Here M stands for the circle in the case of closed curves and for the interval [0, 27r] 
for open curves. 

The space Imm(M, is a smooth infinite dimensional Frechet manifold with tan¬ 
gent space rcIinm(M, the set of all vector fields along the curve c. Using the 
trivialization of we can identify the tangent space with 

Tc Imm(M, := {/i € (M, } . (2) 

On the manifold Imm(M, we consider reparametrization invariant metrics Gc, 
i.e., Riemannian metrics that satisfy 

Gc{h,k) = Gcoipiho ip,ko ip) for all S Diff(M). (3) 

Here, DifF(M) denotes the group of orientation preserving diffeomorphisms of M, 
which consists of all reparametrizations of the curves under consideration. 

The most prominent example of such a metric is the elastic metric G°‘’^ that is 
defined by 


G‘^/ih,k)= [ a\D,h)^\\{Dsk)^\ + b\D,h)^\\{Dsk)^\ds. (4) 

J M 

Here a^, are positive constants, Ds = T^^de denotes the arc length derivative, ds = 
\c'\d9 is arc length integration, v = DgC is the unit length tangent vector, \{Dsh)^\ = 
{Dsh,v) denotes the tangential component of Dgk and \{Dsh)-^\ = Dgh — \{Dsh)^\v 
the normal component. Note that constant vector fields h are in the kernel of 
thus 0 defines only a metric on the manifold of immersions modulo translations. 

Other metrics that have been introduced include higher order Sobolev metrics, i.e., 
metrics of the form 


Giih,k) 



( 5 ) 


with coefficients aj possibly depending on the foot point c. Depending on the order 
of the metric G, local and global well-posedness of the geodesic equation have been 
shown and the metric completions of the corresponding spaces have been studied [5]. 

In the experimental part of this article we will focus on one particular member of 
this family, namely the elastic metric that corresponds to the parameters a = 1, 6 = |. 
This metric has the advantage that it has a very helpful representation as a pullback of 
the flat L^-metric. To see this we introduce the so-called square root velocity transform 
(or short SRVT): 


R : 


Imm(M, M.‘^) 
c 


^ G°°{M, {0}), 

c' 


( 6 ) 
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The SRVT, when regarded for curves modulo translations, has an inverse, which is 
given by: 


R- 


'C'“(M,K^\ {0}) h^Imm(M, 


\q\qd0. 


(7) 


On C°° we can consider the flat L^-metric. In it has been shown, that 

the pullback via R of the L^-metric is exactly the elastic metric Q. The situation for 
open curves is particularly easy, as one has then explicit formulas for geodesics and 
geodesic distance: 

Theorem 1. The image of the R-map of the manifold of open curves is an open subset 
o/C-([0,27r], 


Im(R) = {gGC~([0,27r],K''):|g|^0}. 


( 8 ) 


Two open curves cq, ci can be connected by a geodesic if and only if there exist no 
9 G [0,27r] and no \ > 0 such that Cq{ 9) = —Xc'i{9). In this case, the unique geodesic 
connecting them is given by 


c{t,9) = R ^ {{1 - t)R{co) + tR{ci)). 

Moreover, the induced geodesic distance on lmm([0, 27r],IR‘^) is given by 


d{co,ci) = ||i?(co) -i?(ci)||| 


M9. 


(9) 


( 10 ) 


For a proof of this theorem see [3]. The situation for closed curves is less explicit. We 
have the following characterization of the image of the SRVT, which will build the 
fundament of our algorithms: 


Theorem 2. The image i?(Imm(S'^, K'^)) of the manifold of closed curves under the 
SRVT-transform is a codimension d submanifold of the flat space C°° {S^ . A basis 

of the orthogonal complement (Tgi?(Imm(5'^, is given by the d vectors 


( 11 ) 




Using this basis, efficient numerical methods for calculating geodesics between closed 
curves have been developed, see EH El. 


2.2 The matching functional on the space of parametrized curves 


To define our similarity measure on the product space Imm(M, IR“) x M", we will first 
introduce an energy functional that is defined for arbitrary paths in Imm(M, K'^). We 
will then define the similarity measure as the minimal energy over all paths with given 
boundary shapes [cq] and [ci]. The important features of the similarity measure on 
the space of unparametrized curves will be: 
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(a) The similarity measure does not depend on the choice of representatives {ci,0i) 
of the observed shapes ([c^], C®). 

(b) The optimal deformation is guided by both the shape of the boundary curves 
and by the feature point information. 

(c) The similarity measure forces an exact matching of the unparametrized curves, 
but only an inexact matching of the feature point information. 

Given a curve c G Imm(M, we denote in the following by c(0) the whole vector 
of points c(0*) G 1 < i < n. 

For parameter values ©o = (^o)* ^ feature points Ci = {C\)i G (R'^)" we 

define the energy functional for a given path c: [0,1] —?► Imm(M, R'^) as: 

£(0o,Ci)(c) = / G,(ct,ct)* + AFM(c(l,0o),Ci) . (12) 

Jo 

Here Gc(-, •) is any reparametrization invariant metric on Imm(M, R'^) and FM denotes 
some similarity measure on The only conditions on FM we impose at the 

moment are that FM is lower semi-continuous and FM(C, C) = 0. The first condition 
is necessary for the subsequent energy minimization, while the second condition implies 
that constant paths actually have zero energy. When we discuss later the actual 
computation of energy minimizing paths, we will introduce further conditions that 
make their numerical approximation possible. 

Lemma 3. The energy functional 111 satisfies the invariance property 

£(^-i(0o), Ci)(c o = £-(00, Ci)(c). (13) 

Remark 4- The meaning of this invariance property will become clear in Sect. |2.3[ 
where we will consider the action of the diffeomorphism group on the quotient space 
Imm(M,R‘^) X M". ■ 


Proof. Using the reparametrization invariance of the metric Gc we calculate 

Ci)(c o(p) = f Gcov>((c o ip)^, (c o (p)t)dt AFM ((c o ,^)(1, (p-^{eo)),Ci) 
Jo 

1 

Gc{ct,ct)dt + AFM (c(l, 0o), Ci) = £{9q, Ci)(c). □ 

Using this energy functional we define our similarity measure on the product space 
Imm(M, R'^) x M" of parametrized curves with feature points as 



dp ((co, 9o), {ci,9i)) 


inf £{9q,ci{6i)){c) 

c:[0,l]—>-Imm 


(14) 


where the infimum is taken over all paths c in Imm(M, R'^) that satisfy 


c(0, •) = Co and c(l, •) = ci . 


(15) 
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Remark 5. We do not call the similarity measure a distance, since it is not symmetric 
in general. However, it would be straightforward to construct a symmetric version of 
this. This will be described in Sect. |2.4[ ■ 

Because we fix the endpoint c(l, •) = Ci, we can write the similarity measure dp as 


dp{{co, 6o), {ci,Oi)) 


inf 

c:[0,l]—>-Imm 


5 Ct) dt 


+ AFM(ci(eo),ci(0i)). 


That is, we only minimize the first term of the energy functional, and we do not allow 
to change the value of the second term at all. The meaning of the second term, will 
become clear when we consider it on the shape space of unparametrized curves. 


2.3 The similarity measure on the shape space of unparametrized, 
feature curves. 

In this section we want to use the previously defined similarity measure on parametrized 
curves to induce a similarity measure on the shape space of unparametrized curves with 
feature point information. Therefore we have to determine the induced action of the 
diffeomorphism group on the product space Imm(M, x M". On the first factor 
Imm(M, it is simply given by composition from the right. To compute the action 
on the second factor M" we need to compute the effect of a reparametrization on the 
feature points. We have: 

= co{6l) = (16) 


Thus the induced action on the product space is given by 

{co,6o)oip= {co o (p,ip-^{eo)). 


(17) 


Using the invariance of our similarity measure - cf. Lemma[^- we obtain the following 
result: 


Theorem 6. The similarity measure (14) on Imm(M, M®) x M" induces a similarity 
measure on the shape space of unparametrized curves with additional feature point 
information. The induced functional is given hy: 


d(([co],Co),([ci],Ci)) 


inf dp {{co,9o),ici ° <P,T ■ 

i^6Diff(M) 


(18) 


Here {cj,Oj) are arbitrary representatives of the shapes ([cj],Cj). 

Remark 7. Note, that the energy functional will force an exact matching of the un¬ 
parametrized curves, but only an inexact matching of the feature points. The reason 
for this is that we assume the feature points to be an additional manually entered 
information that is possibly error-prone. ■ 
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Proof. We need to show that d does not depend on the actual choice of representatives 
Co and Ci. Any other representatives of [c^] can be written as Ci o ip for some diffeo- 
morphism ip. Since we are minimizing over all possible reparametrizations of ci, the 
functional clearly does not depend on the choice of the representative ci. It remains 
to verify the independence of reparametrizations of cq. Therefore we calculate 

(ci o ip, ip~'^iei)) 

= inf f ^ inf f (V’"^(0o),Ci(0i)) (c)V 

V?GDiff(M) yc:[0,l]—>-Imm J 

where the infimum is taken over all paths c that satisfy the boundary conditions 


c(0, •) = Co o •0, c(l,-) = Cl oy). 


Using the invariance property of the functional - cf. Lemma [^- we can rewrite this as 


inf dp ((co 0 0,0 i(0o)),(ci o(p,(p ^(©i)) 
i/3GDiff(M) 

= inf ( inf f (0o, Ci(ei)) (c o 0“^)^ 

= inf ( inf £ (0o, Ci(ei)) (c)^ 


such that 


C(0,-) =CoO0O0 ^ = Co, C(l, •) = Cl O 1^, 


which concludes the proof. □ 

We note that the similarity measure d can also be written as 


d(([co],Co),([ci],Ci)) 


= inf 


inf 


c/?GDiff(M) yc:[0,l]—>-Imm 


Cf) dt 


UO 


AFM(cioy,(0o),Ci) 


where the inner infimum is taken over all paths c that satisfy the conditions c(0, •) = cq 
and c(l, •) = Cl o ip. 

Remark 8. Due to the invariance with respect to the reparametrization group, all the 
metrics descend to the shape space of unparametrized curves, i.e., they induce a 
metric on the quotient space S := Imm(M, ]R'^)/DifF(M) such that the projection 

TT : Imm(M,Imm(M,K"^)/Diff(M) 


is a Riemannian submersion. For a detailed discussion of this topic we refer to the 
article m- For A = 0 - i.e., no feature point matching - the similarity measure 
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(18) is then given by the induced geodesic distance on the quotient space. If we 
assume existence of a minimizer, it would be given by a horizontal geodesic on the 
top space, the manifold of parametrized curves. For A > 0 minimizers of will 
still be geodesics on Imm(M, however their initial velocity will in general not be 
horizontal anymore. The induced curve on the quotient space Imm(M, Diff(M) 
will thus not be a geodesic. In [2] such curves have been called ballistic curves. ■ 


Remark 9. In certain applications, there exist natural reference parametrizations c^^ref G 
[ci] of the curves one is interested in. For instance, if one deals with skeletal animations 
(for details see Section]^ below), the curves are mappings from a time interval into the 
so-called joint space. In this case, the reference parametrization of a given animation 
uses a uniform frame rate, and reparametrizations correspond to local speed-ups or 
slow-downs. In such a setting, it makes sense to define the feature matching term 
based on the similarity ofjrarameter values rather than the points on the curve. Given 
some distance measure FM on M", this can be achieved in our setting by defining 


FM(Ci,C2) 


™(Coief(Co),cr.Lf(Cl)) 

-|-oo 


if C, e C,,ref(M)", 

else. 


Then the similarity measure d can be written as 


d(([co],Co),([ci],Ci)) 


= inf 


inf 


t/?GDiff(M) yc:[0,l]—>-Imni 


Gci^Ct t ) dt 


Uo 


+ X FM(v?(0o.ref), ^l,ref) ) , 


where we consider in the inner infimum only paths satisfying c(0, •) = co,ref and c(l, •) = 

Cl,ref O 1 ^. g 


2.4 Symmetrization of the feature matching term 

With the definition in the energy is not symmetric with respect to the two shapes, 
because the feature points are treated differently. It is, however, straightforward to 
symmetrize the energy functional by defining 

£sym(0o, Co, 0u Ci)(c) = [ G,{ct,Ct)dt + A(FM(c(1, 0o), Cl) + FM(c(0,0i), Co)). 

This energy functional satisfies the invariance property 

Co, (^-i(ei), Ci)(c O ^) = £{ 60 , Co, 01, Ci)(c) 

for any diffeomorphism ip of M. From this energy functional we obtain a distance on 
the space of parametrized curves with feature points, 

^P,syni ((co, 0o) j (^1: 01)) ■— ^sym ( 00 ; Cq (0o), 01 , Cl (0i)) (c), 

c:[0,l]—limm 
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where the infimum is taken over all paths c such that c(0, •) = Cg and c(l,-) = Ci- 
Now, the invariance property of E implies that 


dp,sym((co O ^ {9 q)) , {ci O ijj ^ ijj ^ (0i))) = dp,sym ((cq , ^o) , (ci, )) , 

whenever -0 is a diffeomorphism of M. This allows, similar to Theorem to define 
a symmetric similarity measure on the shape space of unparametrized curves with 
feature points by 


d(([co],Co)),([ci],Ci)) = inf fi((co,Co\Co)),(ciO(^,(^ ^oci^Ci))). (19) 


¥3GDiff(M) 


The problem of this similarity measure is that the computation of the infimum in (19) 


requires the evaluation of terms of the form o c)" (Ci),Co), which involve 

the inverse of the diffeomorphism Lp. In particular for derivative based optimization 
methods like gradient descent, this poses problems, as they would require in addition 
to also its derivative. For this reason we have used only the non-symmetric 
similarity measure in all the computational examples below. We note, however, that 
discretizations of the symmetric term can, in certain cases, be minimized efficiently 


with an approach based on dynamic programming (see Sect. 3.21. 


3 Matching feature curves with the elastic metric. 


In the following, we will study one particular choice for both the Riemannian met¬ 
ric and the feature matching term. Our choice of the Riemannian metric G on 
Imm(M,is the elastic metric with coefficients a = 1, 6 = |, see equation Q. 
This is particularly beneficial if we work on open curves. In this case, we have an 
explicit formula for the induced geodesic distance of the metric G, cf. Theorem For 
the feature point matching we use the squared £^-norm on the parameter space M 
with respect to some reference curves c^^ref =: cp We will discuss different choices for 
the feature matching term below in Remarks and 

As a direct consequence, we obtain the following formula for the matching functional 
for feature curves: 


16 


Corollary 10. Using the elastic metric G with coefficients a = l,b = ^ and the 
norm error term the similarity measure (18) on the set of open feature curves reads 
as: 


d(([co],Co),([ci],Ci)) 


= inf 
<^GDiff(Si) 





\gffi0+xj2\m) 



Here, {cj,6j) are arbitrary representatives of the shapes {[cj],Cj). 


( 20 ) 


This observation yields the following strategy for solving the feature curves matching 
problem: 
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• Minimize 


II \g,d0 + XY,\^{01) - 0\\\ (21) 

Jo Vlc'ol yJ\c'i\oip ^ 

over ip S Diff([0, 27r]). 

• Calculate the geodesic connecting cq to ci o ip using the explicit formula from 
Theorem [TJ 


Remark 11. Note, that the minimizer of (211 will in general not be a diffeomorphism, 
but will only have a non-negative derivative, cf. |30j To guarantee existence of the 
minimizer in the diffeomorphism group one would need to use a stronger metric on 
Imm(M, see [8]. ■ 

For closed curves the situation is more complicated, since there is no explicit formula 
for the geodesic distance. Thus the matching functional does not simplify and (21) for 
closed curves reads as: 


£l\p) := dist(co, Cl o pf + 1^5(61^) - e\\^. 


( 22 ) 


However, due to the explicit characterization of the image of the SRVT, there are fast 
and efficient ways to numerically calculate the geodesic distance, cf. [13 [g. 

In the following we will present two methods to minimize these functionals: a dy¬ 
namic programming approach and a gradient descent algorithm. 


3.1 A gradient descent approach 

The integral component of the gradient descent algorithm - the variation of the energy 
functional (21) - will be derived in the following lemma. To simplify the exposition 


we introduce the notation 


q-kp := ^/^{qop), 


(23) 


for the action of the diffeomorphism group on the space of SRV-transformed functions. 
Then we have: 

Lemma 12. The variation of £°o in direction 6p is given by: 

r27r 


S£l{^p){S(p) = {qo- qi^(p,S(p'.{qi'k(p) - 2{qi ^ (p)'.S(p)^dd6 

Jo 


+ 2Xj2{m)-0{)6pm- 


(24) 


2=1 


The -gradient of the energy funetional (21) is then given by: 


grad(£:i(:p)) = -(qo 


(qikpY 


9o 


T' / Rrf \ T' 
where 5gi denotes the delta distribution and qj = R(cj). 


2\Y,{p{e) - 


2 = 1 
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Proof. Using the notation qj = R{cj) the Energy functional can be written as 

n 

= ho-qi^pWh + - 6\\^ . 

i^l 

We will calculate the variation of the two parts separately. For the first part we have: 
5 (Iko - <71 * pWh) {Sip) = 5 (iko - Wki o p)\\h') {^p) 

= -2 y (<7o - \/^(gi o p), ° ‘^ + vV^'i o pSp)^dde 

= -2 / (<70 - <?i * <P, -^qi * (^ + (gi * (/?) 5p)^dde . 

Jo 

To read off the L^-gradient we have to integrate by parts the dp' term. Since Sp 
vanishes at the boundary we have 

- 2 y {qo-qi*p, ^qi * p)Rdd0 = 2 J Sp \^{qo -qi*p, j 

= [ Sp{q'o-(qi-kp)','^^ 

Jo P 

p2TT 

+ / Sp{qQ - (gi *173), 

Jo 


V ql*_fp^ 

,/ ! 

(gi*y’)V'- {q\'^p)p'‘ 

p'"^ 

Using that 

/ , {qi-^p)' ^ p" 

we obtain the gradient of the first term: 

(91 *</?)' {q\'^p)p'‘ 


(25) 


grad(|ko-9i*</3|li2) = ( go - gi * < 73 ,-2g) * <73 + 


P 


P 




= ( go - gi * < 73 , 


+ (g),-(gi*<p)',^) 

\ P ! Ri 

(gi *(/3)'\ / , ^ * 

-^- ) + ( 9o - (91 * < 73 ) ,- 7 

P /r^ \ P 

{qi*p)' 

p' 


, , , qi*p 
+ ('?o. 


= - yio 

For the second summand we calculate 

/ n \ n 

<5 E - O'P {Sp) = 2j2{p{9'o) - 9\)Sp{e',) 


2=1 
n p27T 


= 2E/ S,,{9){p{9) - 9l)Sp{0)d9 . 

A _ 1 0 


Putting everything together, the formula for the gradient follows. 


□ 
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Remark 13. Since can be seen as the restriction of to a co-dimension d snb- 
manifold, the gradient of £'^ is simply given by the projection onto the tangent space 
of this submanifold, cf. Theorem ■ 


Using the above formulas, the implementation of the gradient descend algorithm 
is straightforward. In |25j . however, it has been shown that it can be beneficial to 
represent diffeomorphisms (p as the tuple p = If one works with open 

curves the initial value Xq is always zero. Then there is a one to one correspondence 
between p and Denoting ij) = energy functional on open curves reads 

as: 


■?2(V’) = Iko -' 0 -(gi o ( / k^dr))|||2 + I / 
Jo Jo 


We can now also derive the variation of £ in the i/j-representation: 


Lemma 14. The variation of £2 {tp) in direction Sif is given by: 


S£2{mi’) 

9 (a I / ^ 

= -2/ \d'ip(qo-qi*p ,—-— > 

Jo \ \ /h^ 

+ 4A^(93(6»^) - 6»1) / ipStfdr 
2 = 1 "'0 


Proof. For a variation Sip we calculate 


ipSifdT {qo - qi-kip, 2q[ -k (p)^a dO 


(26) 


i5( qo-ip-(qio ifdTfj 

= ‘2'j (<7o - V’-'?io(y o (y tf'^dT^ 

0 0 

— 2'ip.(^q[ o ijj^dTfj y tpS'ipdr'^ ^dO 
y Sip(^qo - ip.qi o (^J ff^drfqioi^j ip'^dT^ 


= -2 


-4 


^271- re 


lO JO 


p0 p0 

ipS-il) drl^qo - fj.qi o ijf^dTfijj.{q[o(^j ij?dryfj^^d9 . 


Using that /p 'tf’^dr = ip we obtain the formula for the first part. For the second 
summand we calculate 


i5f^|y 'if'^dr — Ol \ {S^f) = 2^2(^J 'ip'^dT — 0\^ J 211 : 611 : dr. □ 


" , ret 
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Remark 15. We do note that a gradient descent based algorithm can also be applied 
if the feature matching term FM is not the squared ^^-norm but rather a general 
differentiable function, the only difference being a corresponding modification of the 
last terms in the variations (24) and (26), respectively. For instance, in the case of a 


feature matching term FM defined on the parameter space M" 
becomes 


the last term in (26) 


2A Vi9*™((p(6»o)-6/i) [°^ljSi;dT. 

i=i -^0 


3.2 Dynamic Programming 

As an alternative to the gradient descent method discussed above. Dynamic Program¬ 
ming (DP) is often used to determine a piecewise linear approximation of the optimal 


parametrization. We begin by introducing a local version of the energy functional (21). 


Let X = {tq, ..., tm} be a discretization of the interval [0, 27r]. Given k < i € I and 
I < j GX and a strictly increasing function (p satisfying ip(\k^i]) = we define 




(yp;fc,Z;z,j):= r [dO + X ^ (27) 


In the special case where p is the linear function 


'fk.i-i.oir) :=1 + {t -k) 


i — k^ 


and 




c'l o / j - ^ 

V^|Ci O (pk.l-.ij I ' ^ ^ 


is the corresponding SRV transform of the reparametrized curve, the energy functional 
reduces to 


E{k,l;i,j) := = 


\qo-qk,l■,^,J\l.d9+X ^ 

<j 


(28) 

Denote now by $ the set of all piecewise linear and increasing homeomorphisms 
ip: [0, 27r] —>■ [0,27r] with vertices on the grid X xX. Denote moreover by ^k,i the set 
of all $ satisfying p{k) = 1. Now let for i, j G X 


H{i,j):= min £i{p;0,0;i,j) 


and denote by pij the (any) corresponding minimizer. That is, H{i,j) is the min¬ 
imal energy required for matching the curve segments co|[o,i] and Ci|[o,j] using a 
piecewise linear reparametrization defined on the given grid. In order to find a 
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Figure 1: The dynamic programming algorithm can be sped up massively by only 
considering predecessor indices (fc, 1) close to the current index (i, j) in (29). 


global reparametrization, we need to find i/(27r, 27r) and a corresponding optimal 
reparametrization (p := (P 2 -k, 2 -k- 

Now note that H satisfies the recursion 


-ff(bj) = ., min E{k,l;i,j) + H{k,l), 


(29) 


because of the additivity of Si. Thus (pij is given by 

with(fc,0e argmin E{k,l;i,j) + H{k,l). 
[fk,l{T) T e [0, fcj, k<i, Kj 

(30) 

In practice, this computation consists of two steps. In a first step, we create the 
M X M matrix H inductively while keeping track of the minimizing indices k and 
I (see (29)). In the second step, we determine the function (p by backtracking the 
minimizing indices and using formula (30). 

In order to speed up the computation, we can restrict the set of admissible indices 
in ( [^ and consider only indices fc, I close to i, j. In practice, this corresponds to a 
restriction of the possible slopes of the piecewise linear reparametrization ip. See Fig. 
[^for an example. 

Remark 16. The minimization approach based on dynamic programming can also be 
applied for non-quadratic feature matching terms, as long as they decompose as 


FM(Co, Cl) = ^ FM„(C'o™, CD, 


with FM,-: 


M>o. For this, one only has to replace in (28) the last sum by 

^ FM^(co((^fc,,;.,,(C)),ci(0D)- 
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Note that this can be also used to implement hard constraints on the deviation of the 
feature points by setting 


FMrn{C^,Cr) 


+00 if ||Co"*-Cr|| >d„ 
0 if ||Co--Cr|| 


for some hard bounds dm > 0. 

In addition, the dynamic programming approach readily extends to the symmetriza- 
tion discussed in Secti on |2.4[ again as long as the feature matching terms decompose. 
Here, the last sum in (|28[) has to be replaced by 




<jf 


We stress here that the function ipk,i;i,j is linear on the interval [fc,i], and thus its 
inverse, which appears in the formula above, can be trivially computed. ■ 

4 Applications 

As demonstrated in |10j . shape matching techniques can be applied to certain com¬ 
puter animations such as, for instance, human walking motions. This has uses in the 
entertainment industry (movie and TV production, and especially video games) as 
well as potential biomedical applications. 

We will be working with skeletal animations, where motions are described in terms 
of bones and joints in an approximation of a human skeleton. A typical approach to 
generate such animation data is to use motion capturing methods, where a stuntman’s 
motions are recorded by multiple cameras in a studio. By tracking a multitude of 
points on the stuntman’s body as he moves, the corresponding skeletal animation 
can be recovered. These animations face a number of limitations, however, and often 
require additional postprocessing. We refer to [201 El [H na [ID] for more details and 
examples. 

A skeleton is a directed acyclic graph where vertices and edges represent bones 
and joints, respectively. A joint represents a transformation relationship between two 
bones. In the case of human motions, transformations between bones are restricted to 
rotations. Joints can have one to three degrees of freedom. For example, the knee has 
one degree of freedom while the foot has two, and the shoulder has three. Fig.[^shows 
the skeleton used for our numerical experiment. By representing rotations using Euler 
angles, we can collect all degrees of freedom of all joints in the skeleton as a high¬ 
dimensional torus, which we refer to as joint-space and denote by 


J := 


where d denotes the total number of degrees of freedom in the skeleton. 

An animation is then a function from a time interval to the joint-space so that 

for every point in time we get a pose of the skeleton. 
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Figure 2: This skeleton, which is based on data from the CMU Graphics Lab Motion 
Capture Database [9], was used for the animation experiments. Figure taken 
from [TU] . 


We can now unroll the joint-space torus in and represent animations as parametrized 
curves cq and ci in this space. The shape matching techniques developed previously 
can then be applied to these curves. See [10] for more details. 

In Sect. |4.2[ we will show how feature matching can be successfully used to com¬ 
plement such existing shape matching methods. We will begin, however, with a few 
examples of feature matching for planar curves to demonstrate the effects the addi¬ 
tional feature term has on the curve matching. 

4.1 2d-curves 

As would be expected, adding feature points to shape matching can have a big impact 
on the resulting paths. 

As a first example, we consider the matching of two open curves, the first of which 
has three maxima and minima, while the second only has two (see Fig. [^. Using only 
the elastic matching term without any specihcation of feature points, the resulting 
minimum energy path between the two curves approximately maps the first and the 
last extremum of the first curve to the first and the last extremum of the second curve, 
while the extremum in the middle vanishes slowly. 

Adding feature points, one can change the behavior of the optimal path significantly. 

If, for instance, feature points are set on the last extremum of the first curve but in 
the vicinity of only the last extremum of the second curve, then the optimal path 
tries, during its evolution, to merge the last extrema of the first curve, while its lower 
portion is matched quite closely to the lower half of the second curve (see Fig.[^ upper 
left). Different behaviors follow from different choices of the feature points. 

Fig.j^shows similar behavior for closed curves. When using a purely elastic matching 
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Figure 3: Effect of picking different feature points when matching two shapes. The 
top left figure shows results for shape matching using only an elastic energy 
functional without feature points. The remaining figures show matching 
results for different combinations of feature points. Corresponding markers 
on the left and right are matched, resulting in different paths between the 
given curves. 


term, moving from the first hand-pose to second one seen in Fig. we end up with 
visually unappealing interpolations. By picking to corresponding fingertips on both 
hands as feature points, the algorithm achieves a much more natural looking transition 
from one shape to the other. However, feature points need to be selected carefully, as 
the last row in Fig. shows. By attempting to match for example the thumb in one 
hand shape to the midway point between index and middle finger on the second hand 
shape, we induce a physically implausible interpolation that involves growing a new 
thumb. 

We refer to the supplementary materiaj^for videos demonstrating the differences. 


^Supplementary material available at https://wiki.niath.ntnu.no/optimizatlon/skeletal_ 
animations 
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Elastic reparametrization 




Incorrect feature points 



Figure 4: Effect of picking different feature points when matching two different hand 
shapes. In the top row, no feature points were set. The purely elastic 
matching produces distorted shapes along the geodesic path between the 
two hand shapes. In the middle row, feature points were set to match the 
tips of ring and index fingers correspondingly. This results in more natural 
interpolated shapes. In the bottom row, we see how incorrect feature matches 
cause some fingers to merge and new fingers to grow along the interpolation 
between the two shapes. Corresponding markers on the left and right are 
matched, resulting in different paths between the given curves. Colors along 
the curves indicate parametrization. 
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4.2 Applications to animations 

We now turn to the use of feature point matching for animations. Human animations 
come in an immensely large variety. Walking motions alone can vary in speed, rhythm, 
step length, motions of the arms and so on. Traditional elastic matching methods 
(i.e., without feature points) can be applied to a large number of animations, but can 
sometimes struggle with animations that have large differences in rhythm, for example 
when matching a walking animation to a limping animation. Feature points can be 
used to help with determining an optimal reparametrization to align two animations 
in time. 



Linear 



Elastic reparam. 



No reparam. 




Feature reparam. 


Figure 5: Example of using various methods to interpolate between two walking an¬ 
imations. The blue and orange lines are the trajectories of the left and 
right feet respectively. Note in particular how the two walking animations 
have different numbers of steps and how the various interpolated anima¬ 
tions struggle with that. We have from left to right and top to bottom the 
following methods: linear interpolation of the Euler angles, elastic match¬ 
ing without reparametrization, elastic matching with reparametrization and 
finally elastic and feature point matching with reparametrization. 


Fig. [5] shows an example of using feature point information to aid in animation 
interpolation. The goal is to calculate interpolations between two different walking 
animations. These can be seen in the top row of Fig. The animations differ in the 
number of steps, rhythm and arm motions. In addition, the character in the second 
animation starts walking forward only after a short delay compared to the first. 

The results of four different interpolation schemes (linear interpolation of the Euler 
angles, elastic matching with and without reparametrization and elastic and feature 
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Linear 


No reparam. 



Figure 6: Example of using various methods to interpolate between two walking an¬ 
imations stepping over an obstacle. The blue and orange lines are the 
trajectories of the left and right feet respectively. We have from left to 
right and top to bottom the following methods: linear interpolation of the 
Euler angles, elastic matching without reparametrization, elastic matching 
with reparametrization and finally elastic and feature point matching with 
reparametrization. 


matching with reparametrization) can be seen on the bottom of Fig. The two 
superimposed lines show the trajectories of the left and right feet for each calculated 
interpolation. We can see how especially the varying numbers of steps in the two initial 
animation causes problems for the matching algorithms. 

As feature points we picked the first three times when the left knee moves forward 
and past the right knee. This is already enough information for the shape matching 
algorithm to determine a “good” (i.e., visually convincing) interpolation. 

Similar results can be seen in the second example, which shows two walking ani¬ 
mations stepping over an obstacle of different height as seen in Fig. Again, the 
superimposed lines represent the trajectories of the feet, with noticeable bumps as 
the character steps over the obstacle. As before, we have picked the first three times 
when the left knee moves forward and past the right knee as feature points. With 
this additional information, the feature point matching algorithm manages to produce 
a visually convincing interpolation between the two animations, whereas the other 
matching algorithms fail and produce only garbled results. 

We refer to the supplementary material^ for a video demonstrating the differences. 

® Supplementary material available at https://wiki.math.ntnu.no/optimizatlon/skeletal_ 
animations 
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